Skip to content

Define Python Prelude in Laurel instead of Core#506

Closed
keyboardDrummer wants to merge 50 commits intomainfrom
remy/pythonPreludeInLaurel
Closed

Define Python Prelude in Laurel instead of Core#506
keyboardDrummer wants to merge 50 commits intomainfrom
remy/pythonPreludeInLaurel

Conversation

@keyboardDrummer
Copy link
Contributor

@keyboardDrummer keyboardDrummer commented Mar 3, 2026

Putting this in draft for now, because my impression is that since the current PySpecs are translated to Core instead of Python or Laurel, it is not possible to reliably turn on resolution error reporting for Laurel when using Python.


This PR turns on resolution error reporting in Laurel, and works towards removing the need for the TCore type in Laurel. It conflicts with changes from another in-progress PR, #489. I think that one will need more time to be ready for merging, so I think it makes sense to review/merge this one first. I will happily resolve conflicts in 489 after this is merged.

Changes

  1. In the Python through Laurel pipeline, replace the use of CorePrelude.lean with PythonRuntimeLaurelPart.lean and PythonRuntimeCorePart.lean. The CorePart is needed because Python requires axioms in Core and Laurel does not support axioms. I think we should not add axioms to Laurel. A long term solution would be that the Python pipeline stops relying on axioms.
  2. Turn on reporting of resolution errors
  3. Add --update option to run_py_analyze.sh to update the test expect files (the bytecode offsets in them change often)
  4. Update Laurel grammar to support multiple preconditions

Caveats

  • Because DDM parsing also does a form of resolution, PythonRuntimeCorePart.lean needs a lot of type definitions that are already provided in PythonPreludeInLaurel.lean. Ideally we'd find a way so that's no longer necessary, using something like #strata_parsing_only. @joehendrix is that something we could add? I looked into adding that but it wasn't so simple.

Tested

  1. Updated Python expect files for Laurel because the reported locations are different
  2. No tests for resolution errors added yet. I will do so in a future PR and I expect this will trigger the need for some fixes as well.
  3. Manually used the --update option for run_py_analyze.sh
  4. Update T6_Preconditions with tests that check multiple preconditions. Tests suffer from Call elimination: requires violation diagnostic should point to call site #531
  5. Bonus: Added T16_Datatypes.lean test for Laurel datatypes, something that was missing in previous PRs

@thanhnguyen-aws
Copy link
Contributor

We will use the "PythonLaurelCorePrelude" in this PR #489 for Python-> Laurel translation instead of the "CorePrelude".

@keyboardDrummer
Copy link
Contributor Author

We will use the "PythonLaurelCorePrelude" in this PR #489 for Python-> Laurel translation instead of the "CorePrelude".

Yes, this PR conflicts with that one, but I think that's OK we can resolve the conflicts. I don't mind resolving them in whatever PR gets merged last.

@keyboardDrummer keyboardDrummer marked this pull request as ready for review March 5, 2026 14:46
@keyboardDrummer keyboardDrummer requested a review from a team March 5, 2026 14:46
@keyboardDrummer keyboardDrummer marked this pull request as draft March 5, 2026 14:55
auto-merge was automatically disabled March 5, 2026 14:55

Pull request was converted to draft

@keyboardDrummer keyboardDrummer marked this pull request as ready for review March 5, 2026 16:16
@joehendrix
Copy link
Contributor

Suggestion: Add test cases for quantifier triggers

The PR adds trigger support to forall/exists (LaurelGrammar.st:68-74), which is threaded through 7+ transformation passes. However, the existing quantifier test (StrataTest/Languages/Laurel/Examples/Fundamentals/T14_Quantifiers.lean) only tests trigger-free quantifiers.

Consider adding trigger test cases to quantifiersProgram, e.g.:

procedure testForallWithTrigger() {
    var a: int := 1;
    assert forall(x: int) {x + 0} => x + 0 == x;
}

procedure testExistsWithTrigger() {
    assert exists(x: int) {x * 1} => x == 42;
}

This would exercise the full pipeline: parsing (OptionalTrigger), resolution (trigger scope), heap parameterization, type hierarchy rewriting, formatting, and Core translation (allTr/existTr). Currently the only trigger usage is in PythonRuntimeLaurelPart.lean (datetime_strptime), which is tested indirectly through the Python pipeline — but test_datetime is skipped in the Laurel test runner.

@joehendrix
Copy link
Contributor

Suggestion: Add test case for unsafe destructor resolution

The PR registers unsafe destructors (DT..field!) during resolution (Resolution.lean:488-489):

_ ← defineName p.name (.parameter p) (some $ dt.name.text ++ ".." ++ p.name.text ++ "!")

And the Laurel prelude uses them, e.g. in PythonRuntimeLaurelPart.lean:207:

days_i := IntOrNone..int_val!(days);

However, no Laurel-level test directly exercises this syntax. The new T16_Datatypes.lean tests safe destructors (IntList..head(xs)) but not the unsafe variant. Consider adding a case:

procedure testUnsafeDestructor() {
  var xs: IntList := Cons(42, Nil());
  assert IntList..isCons(xs);
  var h: int := IntList..head!(xs);
  assert h == 42;
}

This would verify that the ! suffix is correctly parsed, resolved, and translated to Core.

Comment on lines +263 to +264
| .Forall p trigger b => do return ⟨.Forall p (← trigger.attach.mapM (fun pv => have := pv.property; rewriteTypeHierarchyExpr pv.val)) (← rewriteTypeHierarchyExpr b), md⟩
| .Exists p trigger b => do return ⟨.Exists p (← trigger.attach.mapM (fun pv => have := pv.property; rewriteTypeHierarchyExpr pv.val)) (← rewriteTypeHierarchyExpr b), md⟩
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines are 172 chars each (hard limit is 100). Extracting the trigger transformation into a let binding also gives Lean the membership proof via destructuring, which means no explicit decreasing_by clause is needed for the trigger.

Suggested change
| .Forall p trigger b => do return ⟨.Forall p (← trigger.attach.mapM (fun pv => have := pv.property; rewriteTypeHierarchyExpr pv.val)) (← rewriteTypeHierarchyExpr b), md⟩
| .Exists p trigger b => do return ⟨.Exists p (← trigger.attach.mapM (fun pv => have := pv.property; rewriteTypeHierarchyExpr pv.val)) (← rewriteTypeHierarchyExpr b), md⟩
| .Forall p trigger b => do
let trigger' ← trigger.attach.mapM fun ⟨t, _⟩ => rewriteTypeHierarchyExpr t
return ⟨.Forall p trigger' (← rewriteTypeHierarchyExpr b), md⟩
| .Exists p trigger b => do
let trigger' ← trigger.attach.mapM fun ⟨t, _⟩ => rewriteTypeHierarchyExpr t
return ⟨.Exists p trigger' (← rewriteTypeHierarchyExpr b), md⟩

@joehendrix
Copy link
Contributor

joehendrix commented Mar 10, 2026

Suggestion: Simplify heapTransformExpr.recurse termination in HeapParameterization.lean

I was looking into shortening a long line, and found the Forall/Exists trigger handling (lines 320-321) and the decreasing_by block (lines 332-339) can be significantly simplified by destructuring exprMd upfront. This gives Lean the structural proofs it needs automatically, eliminating the entire 9-line decreasing_by clause.

The key change is at the top of recurse:

-- Before:
  recurse (expr : StmtExprMd) (valueUsed : Bool := true) : TransformM StmtExprMd := do
    let md := expr.md
    match _h : expr.val with
    ...
    termination_by sizeOf expr
    decreasing_by
      all_goals (simp_wf; try term_by_mem)
      ...

-- After:
  recurse (exprMd : StmtExprMd) (valueUsed : Bool := true) : TransformM StmtExprMd := do
    let ⟨expr, md⟩ := exprMd
    match _h : expr with
    ...
    termination_by sizeOf exprMd

This requires a few follow-on changes in the match arms:

  • StaticCall: use args.attach.mapM (fun ⟨v, _⟩ => recurse v) instead of args.mapM
  • Assign with FieldSelect: destructure directly in the pattern: | [⟨.FieldSelect target fieldName, _fieldSelectMd⟩] =>
  • Assign fallback simplified
  • New and wildcard: return exprMd instead of expr
  • Forall/Exists triggers: use trigger.attach.mapM fun ⟨t, _⟩ => recurse t

Here is the full revised recurse definition:

  recurse (exprMd : StmtExprMd) (valueUsed : Bool := true) : TransformM StmtExprMd := do
    let ⟨expr, md⟩ := exprMd
    match _h : expr with
    | .FieldSelect selectTarget fieldName =>
        let qualifiedName : Identifier := resolveQualifiedFieldName model fieldName
        let valTy := (model.get fieldName).getType.getD (panic! "heapTransformExpr1")
        let readExpr := ⟨ .StaticCall "readField" [mkMd (.Identifier heapVar), selectTarget, mkMd (.StaticCall qualifiedName [])], md ⟩
        -- Unwrap Box: apply the appropriate destructor
        return mkMd <| .StaticCall (boxDestructorName valTy.val) [readExpr]
    | .StaticCall callee args =>
        let args' ← args.attach.mapM (fun ⟨v, _⟩ => recurse v)
        let calleeReadsHeap ← readsHeap callee
        let calleeWritesHeap ← writesHeap callee
        if calleeWritesHeap then
          if valueUsed then
            let freshVar ← freshVarName
            let varDecl := mkMd (.LocalVariable freshVar ⟨.TInt, #[]⟩ none)
            let callWithHeap := ⟨ .Assign
              [mkMd (.Identifier heapVar), mkMd (.Identifier freshVar)]
              (⟨ .StaticCall callee (mkMd (.Identifier heapVar) :: args'), md ⟩), md ⟩
            return ⟨ .Block [varDecl, callWithHeap, mkMd (.Identifier freshVar)] none, md ⟩
          else
            return ⟨ .Assign [mkMd (.Identifier heapVar)] (⟨ .StaticCall callee (mkMd (.Identifier heapVar) :: args'), md ⟩), md ⟩
        else if calleeReadsHeap then
          return ⟨ .StaticCall callee (mkMd (.Identifier heapVar) :: args'), md ⟩
        else
          return ⟨ .StaticCall callee args', md ⟩
    | .InstanceCall callTarget callee args =>
        let t ← recurse callTarget
        let args' ← args.mapM (recurse ·)
        return ⟨ .InstanceCall t callee args', md ⟩
    | .IfThenElse c t e =>
        let e' ← match e with | some x => some <$> recurse x valueUsed | none => pure none
        return ⟨ .IfThenElse (← recurse c) (← recurse t valueUsed) e', md ⟩
    | .Block stmts label =>
        let n := stmts.length
        let rec processStmts (idx : Nat) (remaining : List StmtExprMd) : TransformM (List StmtExprMd) := do
          match remaining with
          | [] => pure []
          | s :: rest =>
              let isLast := idx == n - 1
              let s' ← recurse s (isLast && valueUsed)
              let rest' ← processStmts (idx + 1) rest
              pure (s' :: rest')
          termination_by sizeOf remaining
        let stmts' ← processStmts 0 stmts
        return ⟨ .Block stmts' label, md ⟩
    | .LocalVariable n ty i =>
        let i' ← match i with | some x => some <$> recurse x | none => pure none
        return ⟨ .LocalVariable n ty i', md ⟩
    | .While c invs d b =>
        let invs' ← invs.mapM (recurse ·)
        return ⟨ .While (← recurse c) invs' d (← recurse b false), md ⟩
    | .Return v =>
        let v' ← match v with | some x => some <$> recurse x | none => pure none
        return ⟨ .Return v', md ⟩
    | .Assign targets v =>
        match targets with
        | [⟨.FieldSelect target fieldName, _fieldSelectMd⟩] =>
          let qualifiedName : Identifier := resolveQualifiedFieldName model fieldName
          let valTy := (model.get fieldName).getType.getD (panic! "heapTransformExpr2")
          let target' ← recurse target
          let v' ← recurse v
          -- Wrap value in Box constructor
          let boxedVal := mkMd <| .StaticCall (boxConstructorName valTy.val) [v']
          let heapAssign := ⟨ .Assign [mkMd (.Identifier heapVar)]
            (mkMd (.StaticCall "updateField" [mkMd (.Identifier heapVar), target', mkMd (.StaticCall qualifiedName []), boxedVal])), md ⟩
          if valueUsed then
            return ⟨ .Block [heapAssign, v'] none, md ⟩
          else
            return heapAssign
        | fields =>
            let targets' ← fields.mapM (recurse ·)
            return ⟨ .Assign targets' (← recurse v), md ⟩
    | .PureFieldUpdate t f v => return ⟨ .PureFieldUpdate (← recurse t) f (← recurse v), md ⟩
    | .PrimitiveOp op args =>
      let args' ← args.mapM (recurse ·)
      -- For == and != on Composite types, compare refs instead
      match op, args with
      | .Eq, [e1, _e2] =>
        let ty := (computeExprType model e1).val
        match ty with
        | .UserDefined _ =>
          let ref1 := mkMd (.StaticCall "Composite..ref!" [args'[0]!])
          let ref2 := mkMd (.StaticCall "Composite..ref!" [args'[1]!])
          return ⟨ .PrimitiveOp .Eq [ref1, ref2], md ⟩
        | _ => return ⟨ .PrimitiveOp op args', md ⟩
      | .Neq, [e1, _e2] =>
        let ty := (computeExprType model e1).val
        match ty with
        | .UserDefined _ =>
          let ref1 := mkMd (.StaticCall "Composite..ref!" [args'[0]!])
          let ref2 := mkMd (.StaticCall "Composite..ref!" [args'[1]!])
          return ⟨ .PrimitiveOp .Neq [ref1, ref2], md ⟩
        | _ => return ⟨ .PrimitiveOp op args', md ⟩
      | _, _ => return ⟨ .PrimitiveOp op args', md ⟩
    | .New _ => return exprMd
    | .ReferenceEquals l r => return ⟨ .ReferenceEquals (← recurse l) (← recurse r), md ⟩
    | .AsType t ty =>
        let t' ← recurse t valueUsed
        let isCheck := ⟨ .IsType t' ty, md ⟩
        let assertStmt := ⟨ .Assert isCheck, md ⟩
        return ⟨ .Block [assertStmt, t'] none, md ⟩
    | .IsType t ty => return ⟨ .IsType (← recurse t) ty, md ⟩
    | .Forall p trigger b =>
        let trigger' ← trigger.attach.mapM fun ⟨t, _⟩ => recurse t
        return ⟨ .Forall p trigger' (← recurse b), md ⟩
    | .Exists p trigger b =>
        let trigger' ← trigger.attach.mapM fun ⟨t, _⟩ => recurse t
        return ⟨ .Exists p trigger' (← recurse b), md ⟩
    | .Assigned n => return ⟨ .Assigned (← recurse n), md ⟩
    | .Old v => return ⟨ .Old (← recurse v), md ⟩
    | .Fresh v => return ⟨ .Fresh (← recurse v), md ⟩
    | .Assert c => return ⟨ .Assert (← recurse c), md ⟩
    | .Assume c => return ⟨ .Assume (← recurse c), md ⟩
    | .ProveBy v p => return ⟨ .ProveBy (← recurse v) (← recurse p), md ⟩
    | .ContractOf ty f => return ⟨ .ContractOf ty (← recurse f), md ⟩
    | _ => return exprMd
    termination_by sizeOf exprMd

Verified: lake build Strata.Languages.Laurel.HeapParameterization passes.

@joehendrix
Copy link
Contributor

Request: Track disabled procedure multiple-requires test

In StrataTest/Languages/Laurel/Examples/Fundamentals/T6_Preconditions.lean:52-57, the procedure version of the multiple-requires test is commented out:

// This test fails because Core incorrectly report error locations on procedure preconditions
// procedure multipleRequiresCaller() {
//  var a: int := multipleRequires(1, 2);
//  var b: int := multipleRequires(-1, 2);
// error: assertion does not hold
// }

Could you create a tracking issue for the Core error location bug so this doesn't get forgotten? The function version (funcMultipleRequires) works correctly, so the issue is specific to procedure precondition error reporting.

@joehendrix
Copy link
Contributor

Question: Is the Laurel pipeline's use of CorePrelude intentional?

The docstrings in both PythonRuntimeCorePart.lean:31 and PythonRuntimeLaurelPart.lean:22 state:

The original CorePrelude.lean remains unchanged for the Python-through-Core pipeline.

However, CorePrelude.lean is also still used in the Laurel pipeline — buildPySpecPrelude starts from Strata.Python.Core.prelude, and the resulting Core.Program is passed to pythonToLaurel (StrataMain.lean:429), which calls extractPreludeTypes and extractPreludeProcedures on it to populate the translation context.

Is this intentional, or is the plan to eventually replace that usage with data from PythonRuntimeLaurelPart/PythonRuntimeCorePart as well? If intentional, the docstrings should note the dual usage. If it's meant to be replaced, might be worth a TODO comment.

@joehendrix
Copy link
Contributor

Per the caveat, the DDM is designed to resolve types to variables as part of its typechecking and type inference. If it doesn't do that, then tradeoff is lack of type inference in polymorphic functions.

It sounds like you don't care about that feature, but it's definitely a feature that was added intentionally by request.

@keyboardDrummer keyboardDrummer marked this pull request as draft March 12, 2026 14:41
auto-merge was automatically disabled March 12, 2026 14:41

Pull request was converted to draft

@keyboardDrummer
Copy link
Contributor Author

keyboardDrummer commented Mar 13, 2026

Request: Track disabled procedure multiple-requires test

Tracked in Asana but sadly not in GH. How do you think we should approach this? Should it be duplicated, only in GH, or is only Asana OK?

@keyboardDrummer
Copy link
Contributor Author

Per the caveat, the DDM is designed to resolve types to variables as part of its typechecking and type inference. If it doesn't do that, then tradeoff is lack of type inference in polymorphic functions.

It sounds like you don't care about that feature, but it's definitely a feature that was added intentionally by request.

It's not a feature that I'm using, and I'm not sure how I would use it. It looks like it performs checks during parsing, but generally the resolution information is not maintained after that. Laurel needs the reference=>definition map for its phases, and it needs to rerun resolution after each phase. What's your intention of how DDM resolution features should be used?

Currently Laurel's resolution logic is specified in Resolution.lean. I expect that from the semantic definition of Laurel, we'll be able to extract the same logic so we won't need much of the contents of Resolution.lean any more. It would then be duplicative to also specify resolution logic in the DDM specification of Laurel. My feeling is that it's good to keep the syntactic and semantic definition of a language separate, so I'm not optimistic about using resolution features of the DDM. What are your thoughts on that?

@keyboardDrummer
Copy link
Contributor Author

Thanks for the excellent review! I've applied the changes to the two PRs replacing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants